{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3d93e4ef-cdca-41ce-8a14-c309293e1b7a",
   "metadata": {},
   "source": [
    "# Note on the Notebook\n",
    "\n",
    "The examples in this notebook are inspired by real-world scenarios I've observed or experienced across different projects. They touch on several of the points discussed during the PoC to Production talk.\n",
    "\n",
    "Each example is designed to:\n",
    "- Work within the limitations of the free Gurobi restricted license, and  \n",
    "- Fit within the time constraints of our session.\n",
    "\n",
    "While the datasets and models here are much smaller than those used in production, the lessons, best practices, and pitfalls to avoid remain exactly the same.\n",
    "\n",
    "In every example that follows, something is wrong.  \n",
    "Your mission, should you choose to accept it, is to identify the culprit(s) using what you've learned so far.\n",
    "\n",
    "**Happy debugging!**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2b6db2b-1034-4d96-92ad-e7ecc5d5f423",
   "metadata": {},
   "source": [
    "# Install Required Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c389b5b6-0aab-4686-9009-8355245b2a71",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install gurobipy\n",
    "%pip install pandas\n",
    "# %pip install plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da02e9c4-cb70-441f-9890-cd6ba0f40a4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import packages\n",
    "import math\n",
    "import random\n",
    "import pandas as pd\n",
    "import gurobipy as gp\n",
    "from gurobipy import GRB\n",
    "import plotly.graph_objects as go"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59d2d0d9",
   "metadata": {},
   "source": [
    "## Set a path to the data used\n",
    "**If you downloaded the notebook/data and are running this locally**, make sure the data is in the same directory as the notebook and run the cell below, or update it to the correct path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "872d3f25",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell if you're working locally on your machine.\n",
    "# Change the path if needed.\n",
    "path = ''"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dbb2e75",
   "metadata": {},
   "source": [
    "**If you are running this in Colab** run this cell below to set the path to read the data from the GitHub repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8d1bdbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell if you are using Colab.\n",
    "path = 'https://raw.githubusercontent.com/Gurobi/modeling-examples/master/optimization301/PoC_to_Production_Exercise/'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b1d2eb8-8875-4eec-8fca-001285136186",
   "metadata": {},
   "source": [
    "# Example 1: Assign Items to Resources\n",
    "Replace *items* with orders, goods, products, groceries, food, or workers, and *resources* with trucks, buses, baskets, or boxes, and you'll start seeing this problem everywhere.\n",
    "\n",
    "Any time you have a limited number of resources with finite capacity, and you need to place, match, assign, or pack your items into those resources, often with goals such as minimizing the number of resources used, you're dealing with a variation of the _bin packing problem_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96e1ee44-cd58-4593-bbe9-539cefbefb40",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ex1():\n",
    "    # Load the data\n",
    "    orders = pd.read_csv('ex1_orders.csv')\n",
    "    vehicles = pd.read_csv('ex1_vehicles.csv')\n",
    "    num_orders = len(orders)\n",
    "    num_vehicles = len(vehicles)\n",
    "    # Build the model\n",
    "    model = gp.Model('vehicle_loading')\n",
    "    # x[i,t]: 1 if order i assigned to vehicle t\n",
    "    x = model.addVars(num_orders, num_vehicles, vtype=GRB.BINARY, name='x')\n",
    "    # y[t]: 1 if vehicle t is used\n",
    "    y = model.addVars(num_vehicles, vtype=GRB.BINARY, name='y')\n",
    "\n",
    "    # Each order assigned to exactly one vehicle\n",
    "    model.addConstrs((x.sum(i, '*') == 1 for i in range(num_orders)), name='assign')\n",
    "    # vehicle capacity constraints\n",
    "    model.addConstrs(\n",
    "        (gp.quicksum(orders.loc[i, 'weight'] * x[i, t] for i in range(num_orders))\n",
    "         <= vehicles.loc[t, 'capacity'] * y[t] for t in range(num_vehicles)), name='capacity')\n",
    "\n",
    "    # Objective: minimize number of used vehicles\n",
    "    model.setObjective(y.sum(), GRB.MINIMIZE)\n",
    "    model.Params.OutputFlag = 0  # turn off the log\n",
    "    model.optimize()\n",
    "    if model.status == GRB.OPTIMAL:\n",
    "        print(f'Optimal. Best Objective: {model.ObjVal}')\n",
    "    elif model.status == GRB.INFEASIBLE:\n",
    "        print('Infeasible!')\n",
    "    else:\n",
    "        print(f'Model Status: {model.status}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e52e5c20-a0e7-4271-9f27-884dc7f5db6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex1()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c990e8a-7afc-4281-9981-75e412d84087",
   "metadata": {},
   "source": [
    "# Solution 1: \n",
    "THAT'S FOR YOU TO FILL..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a23751c-a2fc-4c1e-9351-84850327871f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "35533a22-16fe-4c7a-a3db-38259127ef0f",
   "metadata": {},
   "source": [
    "# Example 2: Unexpected Cost\n",
    "You're working on a problem to determine the optimal flow of products between distribution centers (DCs) and customers, given supply and demand, with the goal of minimizing transportation cost.\n",
    "\n",
    "You've been told that the current solution is around \\\\$6M, and the optimal solution of the model is expected not to exceed this value.  \n",
    "However, the model's optimal solution is close to \\\\$8M. What do you think is going wrong?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e2c281e-3c4d-4d73-a8f6-38945522163a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ex2():\n",
    "    dcs_df = pd.read_csv('ex2_dcs.csv')\n",
    "    cust_df = pd.read_csv('ex2_customers.csv')\n",
    "    lanes_df = pd.read_csv('ex2_lanes.csv')\n",
    "    # Built some objects for convenience and speed\n",
    "    dc = dcs_df['dc_id'].tolist()\n",
    "    cust = cust_df['cust_id'].tolist()\n",
    "    arcs = list(lanes_df[['dc', 'cust']].itertuples(index=False, name=None))\n",
    "    supply = dcs_df.set_index('dc_id')['supply'].to_dict()\n",
    "    demand = cust_df.set_index('cust_id')['demand'].to_dict()\n",
    "    cost = lanes_df.set_index(['dc', 'cust'])['cost'].to_dict()\n",
    "\n",
    "    # Model\n",
    "    model = gp.Model('transport')\n",
    "    # x[d,c]: flow from dc d to customer c\n",
    "    x = model.addVars(arcs, lb=0.0, vtype=GRB.CONTINUOUS, name='x')\n",
    "\n",
    "    # supply constraints\n",
    "    model.addConstrs((x.sum(d, '*') <= supply[d] for d in dc), name='supply')\n",
    "    # demand constraints\n",
    "    model.addConstrs((x.sum('*', c) >= demand[c] for c in cust), name='demand')\n",
    "    # Objective: minimize total transport cost\n",
    "    model.setObjective(gp.quicksum(cost[d, c] * x[d, c] for (d, c) in arcs), GRB.MINIMIZE)\n",
    "    model.Params.OutputFlag = 0  # turn off the log\n",
    "    model.optimize()\n",
    "    if model.Status == GRB.OPTIMAL:\n",
    "        print(f'Objective: {model.ObjVal:,.0f}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6e29fe0-fc81-4f1f-8b3a-f7d6791b6eec",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex2()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c66f65f0-ab70-44fd-b9ea-ca1a3a9282a4",
   "metadata": {},
   "source": [
    "# Solution 2:\n",
    "THAT'S FOR YOU TO FILL..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fab6997-5a66-42f6-9663-2d2c5ba12176",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "1b226934-98a3-4571-b317-ca0217ea706e",
   "metadata": {},
   "source": [
    "# Example 3: Why Infeasible?\n",
    "There are situations where a set of tasks share limited resources, meaning some tasks cannot be executed simultaneously because they depend on or compete for the same resource. We can model this as a compatibility graph, where each task is a node and an edge between two tasks indicates that they have no conflict and can run in parallel. If there is no edge between two tasks (i.e., no compatibility), they cannot both be in the solution. The goal is to find the largest subset of mutually compatible tasks; those that can all be performed at the same time without resource overlap. This is known as the _maximum clique problem_.\n",
    "\n",
    "This problem appears in social network analysis (e.g., finding the largest group of mutual friends), telecom, bioinformatics, manufacturing, and many other domains.\n",
    "\n",
    "In the example below, find out why the function returns infeasible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a4cfe5e-139e-44b9-a823-911ecd6538f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ex3():\n",
    "    # Using this to ensure all are working with the same data\n",
    "    n, p, seed = 200, 0.95, 42\n",
    "    random.seed(seed)\n",
    "    edges = set()\n",
    "    for i in range(n):\n",
    "        for j in range(i + 1, n):\n",
    "            if random.random() < p:\n",
    "                edges.add((i, j))\n",
    "\n",
    "    model = gp.Model('max_clique')\n",
    "    model.Params.TimeLimit = 3\n",
    "    model.Params.OutputFlag = 0\n",
    "\n",
    "    x = model.addVars(n, vtype=GRB.BINARY, name='x')\n",
    "    for i in range(n):\n",
    "        for j in range(i + 1, n):\n",
    "            if (i, j) not in edges:  # no edge, no compatibility\n",
    "                model.addConstr(x[i] + x[j] <= 1, name=f'conflict_{i}_{j}')\n",
    "    # Find the maximum set of compatible nodes\n",
    "    model.setObjective(x.sum(), GRB.MAXIMIZE)\n",
    "    model.optimize()\n",
    "    if model.status == GRB.OPTIMAL:\n",
    "        print(f'Optimal. Best Objective: {model.ObjVal}')\n",
    "    else:\n",
    "        print('Infeasible')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9f67074-d986-49ab-8a51-167b4de8799f",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex3()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f66547b-e2f0-4e69-85b3-a4c77336df8a",
   "metadata": {},
   "source": [
    "# Solution 3:\n",
    "THAT'S FOR YOU TO FILL..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36f41f5d-95cf-4a28-a926-59a581e0a0be",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "64b54ac5-210b-412d-a155-6dc49c87c20e",
   "metadata": {},
   "source": [
    "# Example 4: Non-Integer Integers! \n",
    "This small model looks straightforward with two integer variables.\n",
    "But when you look at the optimal values, something seems off. What's going on?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cdceeb6-f74a-45f9-ba6c-3721d13dd44d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ex4():\n",
    "    model = gp.Model()\n",
    "    x = model.addVar(vtype=GRB.INTEGER, ub=1000, name=\"x\")\n",
    "    y = model.addVar(vtype=GRB.INTEGER, ub=1000, name=\"y\")\n",
    "    # max πx + ey\n",
    "    model.setObjective(math.pi * x + math.e * y, GRB.MAXIMIZE)\n",
    "    model.addConstr(math.pi * x + 1 / 3 * y <= 3141.59265)\n",
    "    model.addConstr(x + y <= 1000)\n",
    "    model.Params.OutputFlag = 0\n",
    "    model.optimize()\n",
    "    if model.Status == GRB.OPTIMAL:\n",
    "        print(f'Optimal Solution: {model.ObjVal}')\n",
    "        for v in model.getVars():\n",
    "            print(v.VarName, v.X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a2bd651-dd0d-48bf-8c80-9976db000db3",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex4()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "566308e3-b066-4e87-a4cb-5cb97eb39263",
   "metadata": {},
   "source": [
    "# Solution 4:\n",
    "THAT'S FOR YOU TO FILL..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d4f6e80-9540-410e-9ee5-20ef83dc2539",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "8ff384d2-173a-40da-b53b-4f98ce70a88a",
   "metadata": {},
   "source": [
    "# Example 5: Unexpected Solution\n",
    "We're revisiting Example 2, but with a new constraint.  \n",
    "This time, we want to decide which DCs to open and determine the flow of products from the open DCs to customers, given supply and demand, to minimize total cost (transportation + fixed opening costs).\n",
    "\n",
    "But the optimal solution is totally unexpected. See if you can find out why."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "704e66ff-cab2-439e-869a-63fcf76e4c19",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ex5():\n",
    "    dcs_df = pd.read_csv('ex2_dcs.csv')\n",
    "    cust_df = pd.read_csv('ex2_customers.csv')\n",
    "    lanes_df = pd.read_csv('ex2_lanes.csv')\n",
    "\n",
    "    # Built some objects for convenience and speed\n",
    "    dc = dcs_df['dc_id'].tolist()\n",
    "    cust = cust_df['cust_id'].tolist()\n",
    "    arcs = list(lanes_df[['dc', 'cust']].itertuples(index=False, name=None))\n",
    "    supply = dcs_df.set_index('dc_id')['supply'].to_dict()\n",
    "    dem_max = cust_df.set_index('cust_id')['demand'].to_dict()\n",
    "    dem_min = cust_df.set_index('cust_id')['min_demand'].to_dict()\n",
    "    cost = lanes_df.set_index(['dc', 'cust'])['cost'].to_dict()\n",
    "\n",
    "    # Model\n",
    "    model = gp.Model('facility_location')\n",
    "    # x[d,c]: flow from dc d to customer c\n",
    "    x = model.addVars(arcs, lb=0.0, vtype=GRB.CONTINUOUS, name='x')\n",
    "    # y[d]: 1 if dc d is open\n",
    "    y = model.addVars(dc, vtype=GRB.BINARY, name='y')\n",
    "\n",
    "    # per customer: min_demand[c] <= sum_d x[d,c] <= demand[c]\n",
    "    model.addConstrs((x.sum('*', c) >= dem_min[c] for c in cust), name='min_demand')\n",
    "    model.addConstrs((x.sum('*', c) <= dem_max[c] for c in cust), name='max_demand')\n",
    "\n",
    "    # dc capacity: sum_c x[d,c] <= supply[d] * y[d]\n",
    "    model.addConstrs((x.sum(d, '*') <= supply[d] * y[d] for d in dc), name='capacity')\n",
    "\n",
    "    # Objective: shipping + fixed cost of opening a dc\n",
    "    shipping = gp.quicksum(cost[d, c] * x[d, c] for (d, c) in arcs)\n",
    "    fixed = 10_000 * y.sum()\n",
    "    model.setObjective(shipping + fixed, GRB.MINIMIZE)\n",
    "    model.Params.OutputFlag = 0\n",
    "    model.optimize()\n",
    "    if model.Status == GRB.OPTIMAL:\n",
    "        print(f'Objective value: {model.ObjVal:.0f}')\n",
    "        print(f'Facilities opened: {sum(int(round(y[d].X)) for d in dc)}')\n",
    "        print(f'Total shipped: {sum(x[arc].X for arc in arcs):.0f}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "197f0778-221e-4083-89bf-0092b7c5c804",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex5()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "107307c9-7ac9-43ab-a15f-52e03875efa0",
   "metadata": {},
   "source": [
    "# Solution 5:\n",
    "THAT'S FOR YOU TO FILL..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2b977bb-0b59-4b17-bdc2-bdb9bf463215",
   "metadata": {},
   "source": [
    "# Example 6: Another Unexpected Solution\n",
    "Let's look at a simplified version of the facility location problem. Our goal is to decide how many facilities to open and how much to ship from each one to meet total demand at the lowest possible cost.\n",
    "\n",
    "The model appears correct, and the solution is reported as optimal,  but something isn't right. Can you figure out what's happening?\n",
    "\n",
    "_Hint_: It's not related to `model.Params.Presolve = 0`. We turned presolve off intentionally to make sure the issue shows up. In a small model like this, Gurobi's presolve is smart enough that if it were on, the model would solve without showing the problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "029beb81-b58e-4b51-9226-1e26df53e2d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ex6():\n",
    "    num_facilities = 3\n",
    "    demand = 100\n",
    "    shipping_cost = 1\n",
    "    facility_cost = 5000\n",
    "    M = 1e8\n",
    "\n",
    "    model = gp.Model()\n",
    "    model.Params.Presolve = 0\n",
    "    # Quantity to ship from each facility\n",
    "    x = model.addVars(num_facilities, name=\"ship\")\n",
    "    # Whether or not we open a facility\n",
    "    y = model.addVars(num_facilities, vtype=GRB.BINARY, name=\"open_facility\")\n",
    "    # Satisfy customer demand\n",
    "    model.addConstr(x.sum() == demand)\n",
    "    # Only ship if facility is open\n",
    "    for i in range(num_facilities):\n",
    "        model.addConstr(x[i] <= M * y[i])\n",
    "    # Minimize shipping costs + facility-opening costs\n",
    "    model.setObjective(shipping_cost * x.sum() + facility_cost * y.sum(), GRB.MINIMIZE)\n",
    "    model.optimize()\n",
    "    if model.Status == GRB.OPTIMAL:\n",
    "        for v in model.getVars():\n",
    "            print(v.VarName, v.X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29e8cb39-4b26-40ff-ac35-c2c09f5e042c",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex6()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3569e80c-f82c-4464-acbd-cafdf7902b65",
   "metadata": {},
   "source": [
    "# Solution 6:\n",
    "THAT'S FOR YOU TO FILL..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5495999b-b30d-4a54-9813-1765f0d0447f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c91ede82-b00d-4a87-89a6-e6b2d740329d",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b952b7bb-2c8b-4978-a1bd-5e25ec4ce1b0",
   "metadata": {},
   "source": [
    "# Other Cautionary Tales\n",
    "Here, we share additional examples, cautionary tales, and best practices to keep in your toolkit as you move from simple notebooks toward production-ready code.\n",
    "\n",
    "## Failure After Project Handoff\n",
    "- If you can't have unit tests for your code, at least deliver it with a few small datasets (in text formats like CSV or JSON that are version-controlled) that run successfully with your model. These act as your basic tests.\n",
    "- If someone modifies the model and later reports an issue, you can rerun your small test dataset first to check whether it still passes. This helps you isolate whether the problem lies in the code or in the new data.\n",
    "\n",
    "## Data-Specific Rules\n",
    "- It's rare to build a model that runs only once and on a single dataset. Different datasets may include their own rules or hidden constraints that aren't obvious at first.\n",
    "- Your model might run perfectly on one dataset and fail on another because of these differences.\n",
    "- **Test your model on multiple datasets that capture the diversity of the system you're modeling.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d1745d5f-ba71-4f96-8ddd-84cda954610f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "opti",
   "language": "python",
   "name": "opti"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
